A Systematic Review of Fault Prediction approaches used in Software Engineering

BACKGROUND – The accurate prediction of where faults are likely to occur in code is important because it can help direct test effort, reduce costs and improve the quality of software. OBJECTIVE – To summarise and analyse the published fault prediction studies in order to identify approaches used to build, measure and validate the performance of fault prediction models. METHOD – A systematic literature review of fault prediction in 148 studies published from January 2000 to December 2009. Studies are classified in terms of their context, the variables and methods used to build models as well as how the performance of a model is measured and validated. RESULTS – An increasing number of studies use machine learning approaches to predict where faults are likely to occur in code. These use a wide variety of methods to build models. Fault prediction models are based mainly on static code metrics, change data and previous fault data. The performance of models is measured in a range of ways that makes cross comparison very complex. The external validation of models is rarely demonstrated. Models reporting very high performance measures (eg. over 90%) need to be treated with particular caution. CONCLUSION – The literature in this area is just beginning to mature with a small but growing number of studies appearing that transparently report models built using rigorous techniques; the performance of these models have been measured in credible ways. However, many studies do not present their methods clearly enough to enable the performance of their models to be convincingly demonstrated. A more standardised way of building and reporting on such performance is needed before potential model users can confidently evaluate these studies.